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INTRODUCTION 


The Virginia Plan for Higher Education articulates the objective that the Commonwealth 
will be the best-educated state by 2030. To achieve this objective, Virginia not only must 
increase educational-attainment rates, but also close the gaps in the differing rates of 
attainment that exist across its population and its regions. The effective use and 


communication of data are powerful tools in that effort. 


Virginia’s Current Attainment vs 2030 Goal 


Total 70.5% 


0.6 Total 53.9% 


— 
0.489 O.808 


2017 2030 
=% of working age Virginians with a credential 
a % of working age Virginians with a degree 


For these higher levels of attainment to be achieved, Virginia must address the “value 
proposition” of higher education to students and families and to the Commonwealth. 
Policy makers, educators, families and students need easy access to trusted information 
upon which to make informed decisions about policies, funding and life choices. While 
SCHEV has significant data assets, including the Virginia Longitudinal Data System 
(VLDS), it is imperative that those data support the alignment among education, 
workforce and human capital needs of the Commonwealth and that information on the 
alignment is readily and easily available to the various users.! 

Within this context, in January 2018 the State Council of Higher Education for Virginia 
(SCHEV) formed an ad hoc committee to develop a blueprint to guide the use of data 
policy and data systems maintained by SCHEV as strategic assets in transforming the 


1 This report is a summary of the ad hoc committee’s considerations, findings and proposed actions. See 
the Appendix for additional information on Code of Virginia authority, committee and staff associated 
with its work. 
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lives of Virginians and the Commonwealth through higher education. Already 
committed to the goals of The Virginia Plan, the Council recognized the need to devote 
attention to making SCHEV’s data resources more focused for policy decision-makers 
and other stakeholders. 


ASSESSING SCHEV’S DATA RESOURCES - GAP ANALYSIS 


At the outset of its deliberations, committee members recognized that the effective use 
of data would be more than just the availability of data; it would be succeeding in 
making that data accessible to the variety of audiences SCHEV needs to serve and 
connecting the data to state priorities. 

In order to explore the question of appropriate data accessibility, the ad hoc committee 
created eight questions to guide their work: 


1. How should we use the data we have more effectively to meet the needs of the 
Council and the Commonwealth? 


2. What are the essential policy questions we should ask and how do we use our 
data to inform the answers? 


3. What data collections do we suspend, continue or expand? Are they complete 
and suitable for achieving the goals of The Virginia Plan? 


4. What improvements should we pursue to make our data more accessible and 
transparent to stakeholders? 


5. What are the best means to communicate data sources, findings and reports to 
various audiences? 


6. What other data sources should we consider incorporating into our system? 
7. Are there data elements that currently do not exist that should be created? 


8. What financial resources will be necessary to achieve the outcomes of the data 
blueprint? 


To meet the committee’s objective, committee members noted the need to identify 
stakeholders to engage in the process. The stakeholders identified included a mix of 
individuals that represent students/parents, institutions, policy makers and 


business/economic development. 


The committee undertook an extensive review of SCHEV’s current research website, 


research.schev.edu, which is in its third iteration since 2001. The data published are 
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intended to be authoritative, objective and consistent with good practices developed in 
the institutional research profession. Many of the site’s reports are unchanged in their 
underlying structure since first developed in the 1990s. These represent a consistent 
history of reporting and data use that users rely upon. Further, these reports are 
designed with institutional use in mind, to allow easy access to comparable data that 
differs little from internal reporting. They are not, however, particularly user friendly to 


the occasional user — including students and families, the media, etc. 


Other reports cross institutions. These are value-added reports that are not possible 
without a single collection point of student-level data. The Transfer Feedback and 
Cohort Lifecycle Reports are examples of cross-institutional reports in that student 
movement between institutions is captured and tracked. The Student Success Index, 
since it is agnostic as to where a student graduates, is a specific example of how these 
data are used. Still other reports cross agencies, most notably the reports on the wage 


outcomes of graduates. 


The committee reviewed the principles upon which the SCHEV Research website was 
built: 
e Reporting and data collection must be consistent with state and federal law. 


e Reporting and data use should be consistent with the Code of Ethics developed 
by the Association for Institutional Research. 


e Reporting of data should first do no harm. It should not appear to reference or 
relate to something that it does not. Appropriate disclosures to the limits of the 
data should be readily available. 


e Reporting should create value for the institutions submitting the data. Large 
quantities of data should not be required of institutions that simply disappear 
into a black box. 

e At all times we should be aware that what we are reporting reflects actions and 
choices of students, and that the policies that are created and enacted based on 
these data are intended to have direct impact on students, thus care should be 
taken to ensure that any reporting or calculations are made accurately and 
meaningfully reflect the underlying data. 


The committee directed staff to complete a survey and interviews of SCHEV data users, 


policy makers and other stakeholders. The committee further directed staff to conduct a 
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review of other state higher education websites and provide examples of best and 


promising practices. 


In February staff developed and published a short survey. A total of 128 stakeholders 
provided responses. An additional 11 Council members and “influencers” (Council 
members and individuals in the legislative and executive branch having significant 
influence on higher education policy) provided individual insights. 


In response to the question, “What are the 


most pressing issues facing Virginia ; 
P 8 "2 8 Two-thirds of the respondents 
Higher Education for which more data and . : : » 
a , said the “user experience 
communications are needed?” the top four 


needed to be improved. 
areas were student success, value, state 


funding and post-graduation outcomes. 
Similarly, in response to the question, “What should SCHEV be studying and reporting 
about higher education in Virginia?”, the top responses were post-graduation outcomes 


and student success. 


Looking at the website, respondents were asked to suggest improvements. Two-thirds 
of the respondents said the “user experience” needed to be improved. The objectivity 
and reliability of the data provided on the website were highly rated by respondents. 
The clarity of tables and graphics received the lowest ratings among users, though 
SCHEV’s reports had more positive rankings than those of other data sites (e.g., 
Integrated Postsecondary Educational Data System Data Center, National Center for 


Higher Education Management Systems, College Navigator, College Scorecard). 


When asked how respondents would like to see data communicated, the most frequent 
response was dashboards, social media notifications, subscriptions and email/data 
newsletters. Surveys and interviews confirmed that SCHEV has many different 
audiences: students and parents looking for financial aid, tuition and fees, debt and job 
outcomes; policy makers looking for institutional performance/outcomes; employers 


looking for workers. That necessitates making the website and data more user friendly. 
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Interviews with “influencers” revealed a view that SCHEV’s data are plentiful, but 
opaque. (“Influencers” are defined as legislative and executive staff and others involved 
in state policy development and implementation.) Influencers articulated the need to 
make SCHEV data more usable for all of its stakeholders. Influencers repeatedly offered 
the view that SCHEV likely has enough data. They felt the pressing issue to be the 
ability to get insights from the data. They suggested the need to start at the end and 
ask, “What questions are we are trying to answer and what data would do that?” And, 
“How do we get the data in a form that would make it easiest for different stakeholders 
to answer those questions?” 


Influencers noted the need to take the 


Influencers noted the need to take ¢Xtensive amount of data SCHEV has and turn 


the extensive amount of data it into useful information for policy makers. 


SCHEV has and turn it into useful Recognizing that policy makers have limited 


‘ ‘ ‘ time and interest, they said the information 
information for policy makers. 


should be put forth in an understandable 


format. Influencers advocated the use of 
multiple layers, such as single page infographics that lead into more extensive analyses, 
so that information is packaged at an appropriate level of detail for the different 
audiences SCHEV should be serving. 


In response to what data SCHEV needed to be reporting, both survey respondents and 


influencers suggested a need for accessible data on: 


e Student success — access, enrollment, retention, graduation, completion, awards. 
e Value — return on investment for the individual and the state. 

e Post-graduation success — employment, wages, quality of life. 

e Student debt — as related to program of study and future wages. 

e Equity and diversity. 

e Cost of education — to the student/family and state support. 


e Availability of and access to financial aid. 
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REVIEW OF OTHER STATE EFFORTS 


Staff provided the committee with a review of other state higher education websites. 
That analysis revealed several major themes on the differences and similarities to 
SCHEV’s data. These major comparisons were organized around the data presentation, 


data accessibility and data reported at comparable state agencies. 


Data Presentation 
In reviewing how other state agencies present their data, a majority of state higher 


education agencies produce static reports on key higher education statistics.” ° “The 
reports are presented typically as an annual comprehensive look at the entire state or on 
specific higher education topics (e.g., enrollment, financial aid). The reports are 
intended to be a digest of information that would be printed and attempt to provide a 


user a comprehensive look at all aspects of higher education in a state. 


Some states publish data in a manner similar to SCHEV’s reports as data visualizations. 
States such as Kentucky°, Minnesota‘, Mississippi’ and Colorado® provide content 
through interactive data visualizations that allow the user to pick a topic of interest and 
gain insight by filtering the data to a certain view. Many of these visualizations operate 
using a data visualization tool, such as Tableau. The visualizations, similar to SCHEV’s, 
guide the user into exploring an organized topic, and then provides the user with the 


flexibility to explore information on the entire topic or on a smaller subset of the data. 


Data Accessibility 
Accessibility of higher education data varies widely based on the presentation the state 


agency uses. Since most states report the state’s data in a format intended for a printed 
format, a major concern identified is the ability to quickly find information or extract 
relevant information from a few limited reports. These static reports require a user to at 


least understand the content available within a report or invest significant resources to 


2 New Mexico Higher Education Department, Data Reports 
3 Indiana Commission Higher Education, Commission Reports 


4 Alabama Commission on Higher Education, Student Database Reports 


5 Kentucky Council on Postsecondary Education, Data 
6 Minnesota Office of Higher Education, Research Data and Reports 


7 Mississippi Institutions of Higher Learning, Interactive Data Portal 
8 Colorado Department of Higher Education, Longitudinal Charts and Graphs 
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learn about the content available. These concerns are diminished with richer data 
visualization products that allow a user to find information easier; however, these data 
visualizations tools employ certain filters and data layouts that may present new issues 


around accessibility for certain individuals. 


An additional concern around the accessibility of data is the ability for stakeholders to 
find data from a state agency. State higher education agencies provide data in a variety 
of ways, but state agencies often lack a centralized location for their specific tools, 
reports or visualizations. Agencies usually provide the same report or data 
visualization in multiple locations on their websites to guide individuals to information. 
This method is slightly different than SCHEV’s as SCHEV’s site has two specific 


locations for SCHEV reports and data visualizations. 


Data Reported 
The data reported by most state agencies intends to provide a snapshot of a few select 


statistics important to understanding the state’s higher education. Common data 
reported by most states include enrollment, financial aid and outcomes. For example, 
Oregon has created “Statewide Higher Education Snapshots” to provide consistent 
displays for each public institution in Oregon and a statewide view.’ The trend to create 
reports or data visualizations on a selected few statistics has allowed states to publish 
infographics that attempt to provide a compact picture of higher education. These 
snapshot views often provide very little contextual information on the data reported, 
and limit the understanding of what additional information could help in decision- 


making. Further, they often implicitly push specific messages. 


° Oregon Higher Education Coordinating Commission, Statewide Higher Education Snapshots 
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Advocacy Partnerships 

In addition to state websites, which tend to be more formal and 
heavily data-driven advocacy partnerships with higher education, 
have developed in a number of states. Their websites provide 
targeted messages, information and links for specific campaigns in 
states. Some examples include: 


Growth4VA 

Growth4VaA is a broad-based bipartisan coalition of business, education 
and political leaders and Virginians from all walks of life who believe that 
higher education is a crucial economic engine for our Commonwealth. 
The coalition was founded by the Virginia Business Higher Education 
Council and supported by all 16 public colleges and universities in 
Virginia, the Virginia Community College System and Virginia’s non-profit 
private institutions. They work actively to promote reform and 
reinvestment throughout the Virginia higher education system.12 


Forward Ohio 

Forward Ohio is a public awareness campaign launched by the Inter- 
University Council of Ohio (IUC), an alliance of Ohio’s 14 public 
universities. Their mission is to promote higher education and highlight the 
ways colleges and universities in Ohio promote growth and opportunity in 
their state.13 See Forward Ohio advocacy reports. 


Complete to Compete 

Mississippi's Complete to Compete (C2C) is a statewide initiative 
designed to help Mississippi adults who have earned some college credit 
- but do not have a degree - better their lives by completing a degree 
program. Created by the Mississippi Institutions of Higher Learning and 
the Mississippi Community College Board, the C2C program combines 
the resources of the state’s 15 community colleges and eight public 
universities.14 


Complete Tennessee 

Complete Tennessee is a nonprofit 501(c)3 education advocacy 
organization focused on increasing postsecondary access and completion 
in Tennessee. Complete Tennessee advocates for students and 
communities by supporting increased postsecondary access and 
completion. Complete Tennessee believes every Tennessean deserves the 
opportunity to earn a postsecondary degree or certificate that leads to 
increased economic mobility, community engagement and a better 
quality of life.15 


Additionally, state agencies 
vary on the data reported. 
The variance in reported 
data are due mainly to their 
individual data collection 
systems or their reliance on 
the Integrated 
Postsecondary Education 
Data System (IPEDS) for 
data. SCHEV’s data 
collection breadth, 
particularly its financial aid 
data is significantly 
different than most other 
states. States such as 
Colorado and Minnesota" 
are reporting on the 
outcomes and wages of 
graduates similar to 
Virginia; however, the 
states reviewed rarely 
report as in depth as 
SCHEV’s data, especially on 
graduate debt and wage 


outcomes. 


Audience is also something 
that varies for reported data 


among states and by tool. 


10 Colorado Department of Higher Education, Postsecondary Degree Earnings Outcomes Tool 
™ Minnesota Office of Higher Education, College Graduate Outcomes 

2 Growth4VA 

13 Forward Ohio 

14 Mississippi Complete to Compete 


15 Tennessee Connect 
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Colorado explicitly states that their graduate earnings outcomes tool is intended to 
inform parents and students. Most other states reviewed either never clarify their 
intended audience or explain their intended audience as anyone in the state with 


interest in higher education data. 


ALIGNING DATA AND INFORMATION WITH THE VIRGINIA PLAN 


Following the reviews of the current status of SCHEV data sources and a comparison to 
other state practices, the committee recommended framing the development of data 
resources to align with The Virginia Plan. Through an iterative process, the committee 
advanced a series of “big questions” in each of the four goal areas of The Virginia Plan, 
recommending the alignment of the website, data and communications around those 


“big questions.” 


Goals - 


AFFORDABLE ACCESS 


What are the financial 
and demographic 
characteristics of 

students who enroll in 

college and those who 
do not? 


Are students enrolling 
in institutions and 
programs for which 
they are appropriately 
prepared? 


How do different paths 
to and through higher 
education compare in 
terms of time, cost 
and outcomes? 


10 
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The committee recommends incorporating 


these questions into the ongoing work of the Through an iterative process, the 
Council. The committee further recommends committee advanced a series of 
that the Council devote the time needed to “big questions” in each of the four 
prioritize the questions and the work goal areas of The Virginia Plan. 


associated with answering them. 


IDENTIFYING FINANCIAL RESOURCES WITH DATA AND POLICY 


Based on the committee’s findings, the Council advanced budget requests aimed at 
improving public awareness of the value of higher education and increased credential 
attainment. The initiatives and potential funding strategies were grounded in the 


committee’s recognition of the: 


e Need for public awareness of the value of higher education: While several 
organizations publish college rankings based on various measures of quality, 
Virginia lacks the means to assess and report on the return on investment that 
higher education provides to individuals, communities and the Commonwealth. 


¢ Need for standard information and coordinated efforts to provide to students 
and parents about postsecondary options: Many entities provide this 
information; however, it comes in many different and uncoordinated forms. 
Many states have launched coordinated efforts to provide basic information for 
all individuals interested in postsecondary education. 


e Need for transparency of information and data: Additional data analysis and 
dissemination of data-informed reports can help students, policy makers, 
institutions and others. 


Recent actions taken to improve data accessibility and relevancy 
Over the past year, SCHEV staff has worked to develop a version of the SCHEV 


Research website. These efforts have focused on making it friendlier to all users, 
creating more appealing visualizations, and developing templates for providing 
answers to the “Big Questions” on the minds of our users. Staff have also worked to 
develop specific pages for potential college students and those that advise them. As part 
of this effort, staff has worked with the Virginia Council of Economic Education on 


these pages to ensure they are accessible and relevant. 
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To address the identified concerns regarding the outcomes of graduates and the return 
on investment of higher education to students, parents and the Commonwealth, the 
Council recommended funding to administer a comprehensive survey of Virginia 
graduates to measure their success in securing employment and contributing to the 
civic life in their communities. The survey results would provide insights for all higher 
education stakeholders, including SCHEV, the Virginia Economic Development 
Partnership, the Governor and the General Assembly. Combined with existing SCHEV 
data, the survey will help policy makers and the public have a better understanding of 


the following issues: 


e How many graduates stay in Virginia? How many out-of-state graduates become 
employed in Virginia? 

e Do they get employment related to their field of study? 

e Are graduates satisfied with their educational experience? 


e What experiences did graduates have in college that were most meaningful to 
their outcomes? 


e How engaged are graduates in civic life? 


In addition to the survey, SCHEV proposed additional resources to increase its capacity 
to analyze and communicate data to policy makers and the general public. SCHEV 
recommended $2 million for these efforts. The General Assembly appropriated $750,000 


to conduct the survey, a one-time expense. 


SCHEV staff has engaged in a contract with 
With these data SCHEV will gain AlumniFinder/Accudata to match graduate 
new insights into where the records with addresses to support the 
graduates of Virginia institutions planned graduate outcomes survey and to 
are living and working. further enhance SCHEV's detailed records on 


wage and debt outcomes. 


SCHEV submitted nearly one million names representing the last 10 years of graduates, 
at all levels, public and Tuition Assistance Grant-participating nonprofit institutions. 
AlumniFinder was able to match addresses to 58% of the graduates and email addresses 
to about 19%, some of which were matched with addresses. With these data SCHEV 
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will gain new insights into where the graduates of Virginia institutions are living and 


working. SCHEV also has a substantial database of potential survey recipients. 


SCHEV has begun creating new visualizations and reports using these data to show the 
potential talent supply by locality in Virginia, as well as earnings by locality. These data 
add needed context and nuance to our reports on wage and debt outcomes because 
geography plays a huge role in earnings. The acquisition of these data are already 
paying dividends in understanding the differences in earnings of the same programs at 


different institutions. 


OPPORTUNITIES TO IMPROVE INFORMATION AND DATA 


From the outset, the ad hoc committee on data and policy was mindful of the roles of 
the Council and of SCHEV as an agency; and as a result, members and staff stayed 
focused on matters of utmost importance in higher education. Although discussions 
expanded into secondary education, economic development and various other topics, 
the findings and potential actions described herein are higher-education-centric and 


involve topics connected to Council’s statutory duties and responsibilities. 


The ad hoc committee offers these initial proposals for further action: 


Need: Public awareness of the value of higher education. 


1. Through a survey of college graduates and other means (e.g., the use of SCHEVs 
data and participation in VLDS), report on the outcomes associated with 
Virginia’s system of higher education. 

2. In coordination with partners, develop and disseminate resources to help 
students and families make good decisions about education beyond high school. 
Successful coalitions, such as Forward Ohio, Mississippi's Complete to Compete, 
and Complete Tennessee offer examples of such partnerships. 

3. Disseminate information on present and future employment opportunities and 
associated degrees and credentials, particularly identifying professions where 
there is a known shortage. 


Need: Transparency and accessibility of information and data. 


1. Revise websites and other communications tools to make them easier to use and 
comprehend. 
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2. Devise the means to gain feedback from users such that improvements can be 
made continuously. 


3. Within security and privacy requirements, seek partnerships with other 
organizations to access and analyze data. 


Need: Alignment of data and information resources with The Virginia Plan for 
Higher Education. 


1. Frame communications and web resources in the context of the “big questions” 
aligned with The Virginia Plan. 


Incorporate effective data use into all facets of the Council’s work. 


Establish the means for SCHEV to support ongoing review and prioritization of 
policy questions and data resources needed to answer them. 


If we fail to increase the transparency and accessibility of Virginia’s higher 
education data resources in this increasingly fast-paced, information-driven, 
technology-rich environment, we will have chosen by default to stand by 


and watch other states move ahead of us. 


CONCLUSIONS 


The needs identified in this report derive from a thorough review of the current status 
of SCHEV data and communications, a scan of other state efforts to provide 
transparent, relevant higher education data to various stakeholders and an 
understanding of the need to take a multi-pronged approach to the data. If the 
Commonwealth commits to the actions and supports these steps with resources over a 
sustained period, we will have provided the tools needed for our citizens to meet The 
Virginia Plan’s goal to become the best-educated workforce in the nation and perhaps 
the world. 


If we aspire to be the best state for business, to be the best-educated state in the nation 
and to ensure opportunity for all Virginians, we must act with urgency to address the 
findings offered in this report. If we fail to increase the transparency and accessibility of 


Virginia’s higher education data resources in this increasingly fast-paced, information- 
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driven, technology-rich environment, we will have chosen by default to stand by and 


watch other states move ahead of us. 


15 


Report of Council’s Ad Hoc Committee on Data and Policy 


APPENDIX A 


The Council, pursuant to § 23.1-203.9 and § 23.1-203.30, Code of Virginia, has 
responsibilities to develop and maintain a comprehensive data information system and 
administer the Virginia Longitudinal Data System. Specifically, SCHEV has the 
responsibilities to: 


9. Develop a uniform, comprehensive data information system designed to 
gather all information necessary to the performance of the Council's duties. The 
system shall include information on admissions, enrollment, self-identified 
students with documented disabilities, personnel, programs, financing, space 
inventory, facilities, and such other areas as the Council deems appropriate. 


30. Administer the Virginia Longitudinal Data System as a multiagency 
partnership for the purposes of developing educational, health, social service, 
and employment outcome data; improving the efficacy of state services; and 
aiding decision-making. 


Duty 9 also includes the following language, which is necessary to protect the privacy 
of individuals for certain data-sharing activities. “When consistent with the 
Government Data Collection and Dissemination Practices Act (§ 2.2-3800 et seq.), the 
Virginia Unemployment Compensation Act (§ 60.2-100et seq.), and applicable federal 
law, the Council, acting solely or in partnership with the Virginia Department of 
Education or the Virginia Employment Commission, may contract with private entities 
to create de-identified student records in which all personally identifiable information 
has been removed for the purpose of assessing the performance of institutions and 
specific programs relative to the workforce needs of the Commonwealth.” 
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APPENDIX B 


Council Ad Hoc Committee on Data and Policy 
Created: January 2018 
Members: Tom Slater (co-chair), Ken Ampy (co-chair), Marge Connelly, Stephen Moret 
Council Participants: Katharine Webb 


Purpose: To develop a blueprint to guide the use of data and data systems maintained 
by SCHEV as strategic assets in transforming the lives of Virginians and the 
Commonwealth through higher education. Through this blueprint, the Council shall: 


: Provide leadership to ensure that Council’s data assets, including the 
Virginia Longitudinal Data System (VLDS), meet the ongoing needs of the 
Commonwealth, are used effectively to communicate with various audiences 
and are adequately funded and staffed. 


: Ensure that Council’s data support the alignment among education, workforce 
and human capital needs of the Commonwealth, and that the Council makes 
information on the alignment readily and easily available to the public. 


: Promote data governance and the protection of individual privacy as a 
fiduciary duty of the Council and its staff. 


° Ensure alignment with The Virginia Plan for Higher Education. 


Goal: Identify needs and recommend actions for improving SCHEV’s data and 
communications resources. 


Staff 


Pam Currey (HCM Strategists and SCHEV) 

Wendy Kang (SCHEV staff) 

Tod Massa (SCHEV staff) 

Marina Moschos (SCHEV staff) 

Kristin Whelan (SCHEV staff) 

Tyler Williams (Virginia Management Fellow) 
Bernadette LeMasters (Virginia Management Fellow) 
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